Denny Wu

About Me

I am a faculty fellow at the Center for Data Science, New York University and the Flatiron Institute.

I obtained my Ph.D. in computer science from the University of Toronto and the Vector Institute, under the supervision of Jimmy Ba and Murat A. Erdogdu. Before that, I was an undergraduate student at Carnegie Mellon University advised by Ruslan Salakhutdinov.

My current research focuses on developing a theoretical understanding (e.g., optimization and generalization) of modern machine learning systems such as neural networks, using tools from high-dimensional statistics.

[CV] [Google Scholar]

Selected Publications

• Emergence and scaling laws in SGD learning of shallow neural networks.
  Yunwei Ren, Eshaan Nichani, Denny Wu, and Jason D. Lee.
  

• Propagation of chaos in one-hidden-layer neural networks beyond logarithmic time.
  Margalit Glasgow, Denny Wu, and Joan Bruna.
  

• Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit.
  Jason D. Lee, Kazusato Oko, Taiji Suzuki, and Denny Wu.
  

• Learning sum of diverse features: computational hardness and efficient gradient-based training for ridge combinations.
  Kazusato Oko, Yujin Song, Taiji Suzuki, and Denny Wu.
  

• Nonlinear spiked covariance matrices and signal propagation in deep neural networks.
  Zhichao Wang, Denny Wu, and Zhou Fan.
  

• Learning in the presence of low-dimensional structure: a spiked random matrix perspective.
  Jimmy Ba, Murat A. Erdogdu, Taiji Suzuki, Zhichao Wang, and Denny Wu.
  

• High-dimensional asymptotics of feature learning: how one gradient step improves the representation.
  Jimmy Ba, Murat A. Erdogdu, Taiji Suzuki, Zhichao Wang, Denny Wu, and Greg Yang.
  

• Convex analysis of the mean-field Langevin dynamics.
  Atsushi Nitanda, Denny Wu, and Taiji Suzuki.
  

• When does preconditioning help or hurt generalization?
  Shun-ichi Amari, Jimmy Ba, Roger Grosse, Xuechen Li, Atsushi Nitanda, Taiji Suzuki, Denny Wu, and Ji Xu.
  

• On the optimal weighted $\ell_2$ regularization in overparameterized linear regression.
  Denny Wu and Ji Xu.
  

• Stochastic Runge-Kutta accelerates Langevin Monte Carlo and beyond.
  Xuechen Li, Denny Wu, Lester Mackey, and Murat A. Erdogdu.
  

Misc.

I like Wes Anderson.

I also like Japanese food.